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1 Introduction 

This chapter is concerned with the design and analysis of algorithms for minimizing finite 
automata. Getting a minimal automaton is a fundamental issue in the use and implementa- 
tion of finite automata tools in frameworks like text processing, image analysis, linguistic 
computer science, and many other applications. 

There are two main families of minimization algorithms. The first by a sequence of re- 
finements of a partition of the set of states, the second by a sequence of fusions or merges 
of states. Among the algorithms of the first family, we mention a simple algorithm de- 



scribed in the book [35 1. It operates by a traversal of the product of the automaton with 
itself, and therefore is in time and space complexity 0(n 2 ). Other algorithms are Hop- 
croft's and Moore's algorithms, which will be considered in depth later. The linear-time 
minimization of acyclic automata of Revuz belongs to the second family. Brzozowski's 
algorithm stands quite isolated and fits in neither of these two classes. 

The algorithm for the minimization of complete deterministic finite state automata 
given by Hopcroft [ [34] ] runs in worst-case time 0(n log n). It is, up to now, the most 
efficient algorithm known in the general case. It has recently been extended to incomplete 
deterministic finite automata [|50|[,[p|]. 

Hopcroft's algorithm is related to Moore's partition refinement algorithm [[I4J, al- 
though it is different. One of the purposes of this text is the comparison of the nature of 
Moore's and Hopcroft's algorithms. This gives some new insight into both algorithms. 
As we shall see, these algorithms are quite different both in behavior and in complexity. 
In particular, we show that it is not possible to simulate the computations of one algorithm 
by the other. 

Moore's partition refinement algorithm is much simpler than Hopcroft's algorithm. It 
has been shown [^| that, although its worst-case behavior is quadratic, its average running 
time is 0(n log n). No evaluation of the average is known for Hopcroft's algorithm. 

The family of algorithms based on fusion of states is important in practice for the 
construction of minimal automata representing finite sets, such as dictionaries in natural 
language processing. A linear time implementation of such an algorithm for cycle-free 
automata was given by Revuz [ |47| ] . This algorithm has been extended to a more general 
class of automata by Almeida and Zeitoun [gp, namely to automata where all strongly con- 
nected components are simple cycles. It has been demonstrated in [^] that minimization 
by state fusion, which is not always possible, works well for local automata. 
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There is another efficient incremental algorithm for finite sets, by Daciuk et al. [27 1. 
The advantage of this algorithm is that it does not build the intermediate trie which is 
rather space consuming. 

We also consider updating a minimal automaton when a word is added or removed 
from the set it recognizes. 

Finally, we discuss briefly the case of nondeterministic automata. It is well-known that 
minimal nondeterministic automata are not unique. However, there are several subclasses 
where the minimal automaton is unique. 

We do not consider here the problem of constructing a minimal automaton starting 
from another description of the regular language, such as the synthesis of an automaton 
from a regular expression. We also do not consider devices that may be more space 
efficient, such as alternating automata or two-way automata. Other cases not considered 
here concern sets of infinite words and the minimization of their accepting devices. 

The chapter is organized as follows. The first section just fixes notation, the next 
describes briefly Brzozowski's algorithm. In Section [|, we give basic facts on Moore's 
minimization algorithm. Section || is a detailed description of Hopcroft's algorithm, with 
the proof of correctness and running time. It also contains the comparison of Moore's 
and Hopcroft's algorithms. The next section is devoted to so-called slow automata. Some 
material in these two sections is new. 

Sections [7] and || are devoted to the family of algorithms working by fusion. We 
describe in particular Revuz's algorithm and its generalization by Almeida and Zeitoun, 
the incremental algorithm of Daciuk et al., and dynamic minimization. The last section 
contains miscellaneous results on special cases and a short discussion of nondeterministic 
minimal automata. 



2 Definitions and notation 

It appears to be useful, for a synthetic presentation of the minimization algorithms of 
Moore and of Hopcroft, to introduce some notation for partitions of the set of states. This 
section just fixes this notation. 

Partitions and equivalence relations. A partition of a set E is a family V of nonempty, 
pairwise disjoint subsets of E such that E = [J P. The index of the partition is the 

number of its elements. A partition defines an equivalence relation =-p on E. Conversely, 
the set of all equivalence classes [x], for x G E, of an equivalence relation on E defines 
a partition of E. This is the reason why all terms defined for partitions have the same 
meaning for equivalence relations and vice versa. 

A subset F of E is saturated by V if it is the union of classes of V. Let Q be another 
partition of E. Then Q is a refinement of V, or V is coarser than Q, if each class of Q 
is contained in some class of V. If this holds, we write Q ^ V. The index of Q is then 
larger than the index of V. 

Given two partitions V and Q of a set E, we denote by U = V A Q the coarsest 
partition which refines V and Q. The classes of U are the nonempty sets P D Q, for 
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Pe? and Q G Q. The notation is extended to a set of partitions in the usual way: we 
write V = V\ A • • • A V n for the common refinement of V\, . . . , V n . If n = 0, then V is 
the universal partition of E composed of the single class E. This partition is the neutral 
element for the A-operation. 

Let F be a subset of E. A partition V of E induces a partition V' of F by intersection: 
V' is composed of the nonempty sets PDF, for P S V. If V and Q are partitions of E 
and Q ^ P, then the restrictions V and Q' to F still satisfy Q' ^ V. 

If P and P' are partitions of disjoint sets E and E', we denote by V V T 7 ' the partition 
of U whose restriction to E and E' are 7-* and 7-" respectively. So, one may write 

V= \J{P}. 

Pev 

Minimal automaton. We consider a deterministic automaton A = (Q,i 7 F) over the 
alphabet A with set of states Q, initial state i, and set of final states F. To each state q 
corresponds a subautomaton of A obtained when q is chosen as the initial state. We call 
it the subautomaton rooted at q or simply the automaton at q. Usually, we consider only 
the trim part of this automaton. To each state q corresponds a language L q (A) which is 
the set of words recognized by the subautomaton rooted at q, that is 

L q (A) = {weA*\q-weF}. 

This language is called the future of the state q, or also the right language of this state. 
Similarly one defines the past of q, also called the left language, as the set {id 6 4* 
i ■ w — q}. The automaton A is minimal if L p (A) ^ L q (A) for each pair of distinct states 
p, q. The equivalence relation = defined by 

p = q if and only if L P (A) = L q (A) 

is a congruence, that is p = q implies p ■ a = q ■ a for all letters a. It is called the Nerode 
congruence. Note that the Nerode congruence saturates the set of final states. Thus an 
automaton is minimal if and only if its Nerode equivalence is the identity. 

Minimizing an automaton is the problem of computing the Nerode equivalence. In- 
deed, the quotient automaton A/= obtained by taking for set of states the set of equiva- 
lence classes of the Nerode equivalence, for the initial state the class of the initial state i, 
for set of final states the set of equivalence classes of states in F and by defining the tran- 
sition function by [p] ■ a = [p ■ a] accepts the same language, and its Nerode equivalence 
is the identity. The minimal automaton recognizing a given language is unique. 



Partitions and automata. Again, we fix a deterministic automaton A — (Q,i,F) over 
the alphabet A. It is convenient to use the shorthand P c for Q \ P when P is a subset of 
the set Q. 

Given a set P C Q of states and a letter a, we denote by a~ 1 P the set of states q such 
that q ■ a G P. Given sets P,RcQ and a £ A, we denote by 



(P,a)\R 
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the partition of R composed of the nonempty sets among the two sets 

R n a~ l P = {q€ R\q-a€ P} and R \ a _1 P = {q€R\q-a$P}. 

Note that R \ a _1 P = P n (a _1 P) c = Pn a^ 1 {P c ) so the definition is symmetric in P 
and P c . In particular 

(P,a)\R = (P c ,a)\R. (2.1) 

The pair (P, a) is called a splitter. Observe that (P, a)\R = {R} if either R ■ a C P 
or P • a fl P = 0, and (P, a)|P is composed of two classes if both R ■ a n P 7^ and 
P • a n P c 7^ or equivalently if P • a <£_ P c and P • a <£ P. If (P, a)|P contains two 
classes, then we say that (P, a) sp/ifs P. Note that the pair S = (P, a) is called a splitter 
even if it does not split. 

It is useful to extend the notation above to words. Given a word w and sets P,RcQ 
of states, we denote by w~ 1 P the set of states such that q ■ w G P, and by (P, w)\R the 
partition of P composed of the nonempty sets among 

Rnw~ l P = {qeR \ q-weP} and R \ w~ l P = {q € P | q ■ w £ P} . 

As an example, the partition (P, w) \Q is the partition of Q into the set of those states from 
which w is accepted, and the other ones. A state q in one of the sets and a state q' in the 
other are sometimes called separated by uu. 

The Nerode equivalence is the coarsest equivalence relation on the set of states that is 
a (right) congruence saturating F. With the notation of splitters, this can be rephrased as 
follows. 

Proposition 2.1. The partition corresponding to the Nerode equivalence is the coarsest 
partition V such that no splitter (P, a), with P S V and a € A, splits a class in V, that is 
such that (P, a) \R = {R} for all P,ReP and a e A □ 

We use later the following lemma which is already given in Hopcroft's paper [p4h. It 
is the basic observation that ensures that Hopcroft's algorithm works correctly. 

Lemma 2.2. Let P be a set of states, and let V = {Pi, P2} be a partition of P. For any 
letter a and for any set of states R, one has 

(P, a) \R A (Pi, a) \R = (P, a)\R A (P2, a)\R = (Pi, a)\R A (P2, a)\R, 

and consequently 

(P,a)\R^(P 1 ,a)\RA(P 2 ,a)\R, (2.2) 
(Pi,a)|P>(P,a)|PA(P 2 ,a)|P. (2.3) 



3 Brzozowski's algorithm 

The minimization algorithm given by Brzozowski [|l8[| is quite different from the two 
families of iterative algorithms (by refinement and by fusion) that we consider in this 
chapter. Although its worst-case behavior is exponential, it is conceptually simple, easy 
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a,b a,b 




Figure 1. The automaton on the left recognizing the language A*aA n . It has n + 1 
states and the minimal deterministic automaton for this language has 2" states. The 
automaton on the right is its reversal. It is minimal and recognizes A n aA*. 



to implement, and it is quite efficient in many cases. Moreover, it does not require the 
automaton to be deterministic, contrary to the algorithms described later. 

Given an automaton A = (Q, I, F, E) over an alphabet A, its reversal is the automa- 
ton denoted A R obtained by exchanging the initial and the final states, and by inverting 
the orientation of the edges. Formally ,4^ = (Q, F, I, E R ), where E R = {(p,a,q) \ 
(q, a,p) G E}. The basis for Brzozowski's algorithm is the following result. 

Proposition 3.1. Let A be a finite deterministic automaton, and let A~ be the determin- 
istic trim automaton obtained by determinizing and trimming the reversal A R . Then A~ 
is minimal. 

For a proof of this proposition, see for instance Sakarovitch's book [f48|]. The min- 
imization algorithm now is just a double application of the operation. Observe that the 
automaton one starts with need not to be deterministic. 

Corollary 3.2. Let A be a finite automaton. Then (A~)~ is the minimal automaton of A. 

Example 3.1. We consider the automata given in Figure [ljover the alphabet A — {a, b}. 
Each automaton is the reversal of the other. However, determinization of the automaton 
on the left requires exponential time and space. 



4 Moore's algorithm 

The minimization algorithm given by Moore |Q] computes the Nerode equivalence by a 
stepwise refinement of some initial equivalence. All automata are assumed to be deter- 
ministic. 

4.1 Description 

Let A — (Q, i, F) be an automaton over an alphabet A. Define, for q G Q and h ^ 0, the 
set 

Lf\A) = {w e A*\ \w\ ^ h, q ■ w e F} . 
The Moore equivalence of order h is the equivalence =/j on Q defined by 

p= h q <=► Lf\A)=L^{A). 
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Moore(_4) 
V <r- {F,F C } 
repeat 

for all a 6 A do 

V a ^ f\ PeP (P,a)\Q 

until V = V 



> The initial partition 

> "P' is the old partition, V is the new one 



Figure 2. Moore's minimization algorithm. 



Using the notation of partitions introduced above, one can rephrase the definitions of the 
Nerode equivalence and of the Moore equivalence of order h. These are the equivalences 
defined by 

f\ (F,w)\Q, and f\ (F,w)\Q . 

w£A* w£A", \w\^h 

Since the set of states is finite, there is a smallest h such that the Moore equivalence =t 
equals the Nerode equivalence =. We call this integer the depth of Moore's algorithm 
on the finite automaton A, or the depth of A for short. The depth depends in fact only 
on the language recognized by the automaton, and not on the particular automaton under 
consideration. Indeed, each state of an automaton recognizing a language L represents in 
fact a left quotient u^ 1 L for some word u. 

The depth is the smallest h such that =h equals =h+i- This leads to the refinement 
algorithm that computes successively =o, =i, . . . , =h, ■ ■ ■ , halting as soon as two con- 
secutive equivalences are equal. The next property gives a method to compute the Moore 
equivalences efficiently. 

Proposition 4.1. For two states p,q € Q, and h J? 0, one has 

p =h+i q <=> p =h q and p ■ a =h q ■ a for all a £ A . (4.1) 

We use this proposition in a slightly different formulation. Denote by Mh the partition 
corresponding to the Moore equivalence =/, . Then the following equations hold. 

Proposition 4.2. For h ^ 0, one has 

M h+1 =M h A/\ f\ (P,a)\Q= V (A A ( p > a )\ R )- n 

aeAPeM h ReM h aeAPeM h 

The computation is described in Figure ^. It is realized by a loop that refines the current 
partition. The computation of the refinement of k partitions of a set swith n elements can 
be done in time 0(kn 2 ) by brute force. A radix sort improves the running time to 0(kn). 
With k = Card(A), each tour in the loop is realized in time O(kn), so the total time 
is 0(£kn), where I is the number of refinement steps in the computation of the Nerode 
equivalence =, that is the depth of the automaton. 
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The worst case behavior is obtained for I = n — 2. We say that automata having 
maximal depth are slow and more precisely are slow for Moore automata. These automata 
are investigated later. We will show that they are equivalent to automata we call slow for 
Hopcroft. 

4.2 Average complexity 

The average case behavior of Moore's algorithm has recently been studied in several 
papers. We report here some results given in [|7| ^8j]. The authors make a detailed analysis 
of the distribution of the number I of refinement steps in Moore's algorithm, that is of the 
depth of automata, and they prove that there are only a few automata for which this depth 
is larger than log n. 

More precisely, fix some alphabet and we consider deterministic automata over this 
alphabet. 

A semi-automaton K, is an automaton whose set of final states is not specified. Thus, 
an automaton is a pair (JC, F), where JC is a semi-automaton and F is the set of final states. 
The following theorem given an upper bound on the average complexity of Moore's algo- 
rithm for all automata derived from a given semiautomaton. 

Theorem 4.3 (Bassino, David, Nicaud Let K be a semi- automaton with n states. 
The average complexity of Moore's algorithm for the automata (/C, F), for the uniform 
probability distribution over the sets F of final states, is 0(n log n). 

The result also holds for Bernoulli distributions for final states. The result remains 
valid for subfamilies of automata such as strongly connected automata or group automata. 

When all semi-automata are considered to be equally like, then the following bound 
is valid. 

Theorem 4.4 (David [^8[). The average complexity of Moore 's algorithm, for the uniform 
probability over all complete automata with n states, is 0(n log log n). 

This result is remarkable in view of the lower bound which is given in the following 
statement [[|. 

Theorem 4.5. If the underlying alphabet has at least two letters, then Moore 's algorithm, 
applied on a minimal automaton with n states, requires at least Cl(n log log n) operations. 



5 Hopcroft's algorithm 

Hopcroft [[34j has given an algorithm that computes the minimal automaton of a given 
deterministic automaton. The running time of the algorithm is 0(kn log n) where k is 
the cardinality of the alphabet and n is the number of states of the give n automaton. The 
algorithm has been described and re-described several times [^, |i[ |lOj |l6[ ^j. 
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Hopcroft(^I) 

V <- {F, F c } 

for all a e A do 

ADD((min(F,F c ),a),W) 
while W 7^ do 

(W,a) <- TakeSome(W) 
for each P G P which is split by (W, a) do 
P',P" <- (W,a)|P 
Replace P by P' and P" in V 
for all 6 e A do 
if (P, 6) £ W then 

Replace (P, b) by (P', 6) and (P", 6) in W 
else 

ADD((min(P',P"),fe), W) 



> The initial partition 

> The waiting set 

> Initialization of the waiting set 

> Take and remove some splitter 

> Compute the split 

> Refine the partition 

> Update the waiting set 



Figure 3. Hopcroft's minimization algorithm. 



5.1 Outline 

The algorithm is outlined in the function HOPCROFT given in Figure [3| We denote by 
min(P P') the set of smaller size of the two sets P and P', and any one of them if they 
have the same size. 

Given a deterministic automaton A, Hopcroft's algorithm computes the coarsest con- 
gruence which saturates the set F of final states. It starts from the partition {F, F c } which 
obviously saturates F and refines it until it gets a congruence. These refinements of the 
partition are always obtained by splitting some class into two classes. 

The algorithm proceeds as follows. It maintains a current partition V = {Pi, . . . , P n } 
and a current set W of splitters, that is of pairs (W, a) that remain to be processed, where 
W is a class of V and a is a letter. The set W is called the waiting set. The algorithm stops 
when the waiting set W becomes empty. When it stops, the partition V is the coarsest 
congruence that saturates P. The starting partition is the partition {F, F c } and the starting 
set W contains all pairs (min(P, P c ), a) for a E A. 

The main loop of the algorithm removes one splitter (W,a) from the waiting set W 
and performs the following actions. Each class P of the current partition (including the 
class W) is checked as to whether it is split by the pair (W, a). If (W, a) does not split P, 
then nothing is done. On the other hand, if (W, a) splits P into say P' and P", the class 
P is replaced in the partition V by P' and P". Next, for each letter b, if the pair (P, b) 
is in W, it is replaced in W by the two pairs (P', b) and (P", b), otherwise only the pair 
(min(P', P"), b) is added to W. 

It should be noted that the algorithm is not really deterministic because it has not 
been specified which pair (W, a) is taken from W to be processed at each iteration of 
the main loop. This means that for a given automaton, there are many executions of the 
algorithm. It turns out that all of them produce the right partition of the states. However, 
different executions may give rise to different sequences of splitting and also to different 



10 



J. Berstel, L. Boasson, O. Carton, I. Fagnot 



running time. Hopcroft has proved that the running time of any execution is bounded by 

0(\A\n\ogn). 



5.2 Behavior 

The pair (V, W) composed of the current partition and the current waiting set in some 
execution of Hopcroft's algorithm is called a configuration. The following proposition 
describes the evolution of the current partition in Hopcroft's algorithm. Formula 5.1 is 
the key inequality for the proofs of correctness and termination. We will use it in the 
special case where the set R is a class of the current partition. 

Proposition 5.1. Let (V, W) be a configuration in some execution of Hopcroft's algo- 
rithm on an automaton A on A. For any P £ V, any subset R of a class ofV, and a € A, 
one has 

(P,a)\R> f\ (W,a)\R, (5.1) 

(W,a)eW 

that is, the partition (P, a)\R is coarser than the partition A(Wa)ew(^' a )\R- 

Proof. The proof is by induction on the steps of an execution. The initial configuration 
(V, W) is composed of the initial partition is V = {F, F c } and the initial waiting set W 
is either W = {(F, a) \ a e A} or W = {(F c , a) a e A}. Since the partitions (F, a)\R 
and (F c , a) \R are equal for any set R the proposition is true. 

Assume now that (V, W) is not the initial configuration. Let (V, W) be a configu- 
ration that precedes immediately ("P, W). Thus (V, W) is obtained from (P, W) in one 
step of Hopcroft's algorithm, by choosing one splitter S in W, and by performing the 
required operations on V and W. 



First we observe that, by the algorithm, and by Lemma 2.2, one has for any set of 
states R, 

f\ (W,a)\R> f\ (W,a)\R. (5.2) 

(W,a)eW\{S} (W,a)EW 

Indeed, the set W contains all W \ {S} with the exception of those pairs (P, a) for which 



P is split into two parts, and in this case, the relation follows from (2.2). 

Next, consider a subset R of a set of V. Since R was not split by S, that is since 
S\R = {R}, we have 

f\ (W,a)\R= f\ (W,a)\R. (5.3) 

(W,a)eW (W,a)eW\{S} 

Moreovei\by the induction hypothesis, and since R is also a subset of a set of V, we 
have for any P G V, 

(P,a)\R> f\ (W,a)\R. (5.4) 

(W,a)£W 
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Consequently, for any subset R of a set of V, any P G V, in view of 5.2, 5.3 and 5.4 
have 

(P,a)\R> f\ (W,a)\R. 
(w,o)eW 



(5.5) 



Let now i? G V , a G A, and let again P be a subset of a set of P. We consider the 
partition (P, a)\R. We distinguish two cases. 

Case 1 . Assume P G V. The proof follows directly from (p".5|). 

Case 2. Assume P £ V. Then there exists PeP such that S\P = {P, P'}. 

If (P, a) G VV \ {5}, then both (P, a) and (P', a) are in W. Consequently 

{P,a)\R^ f\ (W,a)\R. 

(W,a)eW 

If, on the contrary, (P, a) ^ W \ then by the algorithm, one of (P, a) or (P', a) 
is in W. 

Case 2a. Assume that (P, a) G W. Then obviously 

(P,a)|P^ /\ (W,a)|P. 

(VK,a)eW 



Case 2b. Assume (P', a) G W. By Lemma 2.2, we have 

(P,a)|P ^ (P', a)\R A (P, a)|P 

as obviously we have 

(P',a)|P^ /\ (W,a)|P. 

(Ea)£W 



and by use of (5.5), we obtain 



{P,a)\R^ f\ (W,a)\R. 

(W,a)eW 



This completes the proof. 



□ 



Corollary 5.2. The current partition at the end of an execution of Hopcroft's algorithm 
on an automaton A is the Nerode partition of A. 



Proof. Let V be the partition obtain ed at the end of an execution of Hopcroft's algorithm 
on an automaton A. By Proposition |2.l[ it suffices to check th at no splitter splits a class 
of V. Since the waiting set W is empty, the right-hand side of (5. 1 ) evaluates to {R} for 
each triple (P, a, R). This means that (P, a) indeed does not split R. □ 



5.3 Complexity 



Proposition 5.3. Hopcroft's algorithm can be implemented to have worst-case time com- 
plexity Oikn log n) for an automaton with n states over a k-letter alphabet. 
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To achieve the bound claimed, a partition V of a set Q should be implemented in a 
way to allow the following operations: 

• accessing the class to which a state belongs in constant time; 

• enumeration of the elements of a class in time proportional to its size; 

• adding and removing of an element in a class in constant time. 

The computation of all splittings P', P" of classes P by a given splitter (W, a) is done 
in time 0(Ca,rd(a~ 1 W)) as follows. 

(1) One enumerates the states q in a -1 W. For each state q, the class P of q is marked 
as a candidate for splitting, the state q is added to a list of states to be removed from 
P, and a counter for the number of states in the list is incremented. 

(2) Each class that is marked is a candidate for splitting. It is split if the number of 
states to be removed differs from the size of the class. If this holds, the states in the 
list of P are removed to build a new class. The other states remain in P. 

The waiting set W is implemented such that membership can be tested in constant 
time, and splitters can be added and removed in constant time. This allows the replace- 
ment of a splitter (P, b) by the two splitters (P' , b) and (P", b) in constant time, since in 
fact P 1 is just the modified class P, and it suffices to add the splitter (P", b). 

Several implementations of partitions that satisfy the time requirements exist. Hop- 
croft p4| ] describes such a data structure, reported in jfl]. Knuutila [ 40 1 gives a different 
implementation. 



Proof of Proposition 5.3 For a given state q, a splitter (W, a) such that q G W is called 
a q-splitter. 

Consider some g-splitter. When it is removed from the waiting set W, it may be 
smaller than when it was added, because it may have been split during its stay in the 
waiting set. On the contrary, when a g-splitter is added to the waiting set, then its size is 
at most one half of the size it had when it was previously removed. Thus, for a fixed state 
q, the number of (/-splitters (W, a) which are removed from W is at most k log n, since at 
each removal, the number of states in W is at most one half of the previous addition. 

The total number of elements of the sets a~ 1 W, where (W, a) is in W, is 0(kn\og n). 
Indeed, for a fixed state q 6 Q, a state p such that p ■ a = q is exactly in those sets a~ 1 W, 
where (W, a) is a g-splitter in W. There are at most 0(log n) of such sets for each letter 
a, so at most (k log n) sets for each fixed q. Since there are n states, the claim follows. 
This completes the proof since the running time is bounded by the size of the sets a~ x W, 
where (W, a) is in W. □ 



5.4 Miscellaneous remarks 

There are some degrees of freedom in Hopcroft's algorithm. In particular, the way the 
waiting set is represented may influence the efficiency of the algorithm. This issue has 
been considered in [^0|. In |^] some practical experiments are reported. In [ p3[ | it is 
shown that the worst-case reported in Jl3| ] in the case of de Bruijn words remains of 
this complexity when the waiting set is implemented as a queue (LIFO), whereas this 
complexity is never reached with an implementation as a stack (FIFO). See this paper for 
other discussions, in particular in relation with cover automata. 




Hopcroft's algorithm, as reported here, requires the automaton to be complete. This 
may be a serious drawback in the case where the automaton has only a few transitions. For 
instance, if a dictionary is represented by a trie, then the average number of edges per state 
is about 1.5 for the French dictionary (personal communication of Dominique Revuz), 
see Table || below. Recently, two generalizations of Hopcroft's algorithm to incomplete 
automata were presented, by [|J| and [|50||, with running time 0(m log n), where n is the 
number of states and m is the number of transitions. Since m ^ kn where k is the size of 
the alphabet, the algorithm achieves the same time complexity. 



5.5 Moore versus Hopcroft 

We present an example which illustrates the fact that Hopcroft's algorithm is not just a 
refinement of Moore's algorithm. This is proved by checking that one of the partitions 
computed by Moore's algorithm in the example does not appear as a partition in any of 
the executions of Hopcroft's algorithm on this automaton. 

The automaton we consider is over the alphabet A = {a, b}. Its set of states is Q = 
{0, 1,2, ... , 9}, the set of final states is F — {7, 8, 9}. The next-state function and the 
graph are given in Figure [| 

The Moore partitions are easily computed. The partition Aii of order 1 is composed 
of the five classes: 

{0,3,4},{1,2},{5,6},{7,9},{8}. 

The Moore partition of order 2 is the identity. 

The initial partition for Hopcroft's algorithm is {F, F c }, where F — 789 (we will rep- 
resent a set of states by the sequence of its elements). The initial waiting set is composed 
of (789, a) and (789, b). There are two cases, according to the choice of the first splitter. 

Case 1. The first splitter is (789, a). Since a _1 789 = 568, each of the classes F and 
F c is split. The new partition is 01234|56|79|8. The new waiting set is 



(79, 6), (8, 6), (8, a), (56, a), (56, b) 
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p 


56 


8 


79 


12 


a~ x P 


49 


6 


58 


017 


b~ x P 


2 


46 


0357 


17 



Table 1. The sets c 1 P, with (P, c) in a waiting set. 

The first three columns in Table [TJ contain the sets c _1 P, for (P, c) in this waiting set. By 
inspection, one sees that each entry in these columns cuts off at least one singleton class 
which is not in the Moore equivalence M.\. This implies that Mi cannot be obtained by 
Hopcroft's algorithm in this case. 

Case 2. The first splitter is (789, b). Since b~ 1 789 = 034568, the new partition is 
12|03456|79|8. The new waiting set is 

(79, a), (8, a), (8, 6), (12, a), (12, b) . 

Again, each entry in the last three columns of Table [l] cuts off at least one singleton class 
which is not in the Moore equivalence M.\. This implies that, also in this case, M.\ 
cannot be obtained by Hopcroft's algorithm. 

Despite the difference illustrated by this example, there are similarities between Moo- 
re's and Hopcroft's algorithms that have been exploited by Julien David in his thesis [ p9| ] 
to give an upper bound on the average running time of Hopcroft's algorithm for a partic- 
ular strategy. 

In this strategy, there are two waiting sets, the current set W and a future waiting set 
T. Initially, T is empty. Hopcroft's algorithm works as usual, except for line 14: Here, 
the splitter (min(P', P"), b) is added to T and not to W. When W is empty, then the 
contents of T and W are swapped. The algorithm stops when both sets W and T are 
empty. 

Proposition 5.4 (David [p9|]). There is a strategy for Hopcroft's algorithm such that its 
average complexity, for the uniform probability over all complete automata with n states, 
is 0(n log log n). 

Julien David shows that at the end of each cycle, that is when W becomes empty, the 
current partition V of the set of states is in fact a refinement of the corresponding level 
in Moore's algorithm. This shows that the number of cycles in Hopcro ft's algorithm, for 
this strategy, is bounded by the depth of the automaton. Thus Theorem |4~4| applies. 



6 Slow automata 

We are concerned in this section with automata that behave badly for Hopcroft's and 
Moore's minimization algorithms. In other terms, we look for automata for which Moo- 
re's algorithm requires the maximal number of steps, and similarly for Hopcroft's algo- 
rithm. 
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6.1 Definition and equivalence 

Recall that an automaton with n states is called slow for Moore if the number I of steps 
in Moore's algorithm is n — 2. A slow automaton is minimal. It is equivalent to say that 
each Moore equivalence =h has exactly h + 2 equivalence classes for h ^ n — 2. This 
is due to the fact that, at each step, just one class of =h is split, and that this class is split 
into exactly two classes of the equivalence =h+i- 



Proposition 4.2 takes the following special form for slow automata. 



Proposition 6.1. Let A be an automaton with n states which is slow for Moore. For all 
n — 2 > h ^ 0, there is exactly one class R in A4h which is split, and moreover, if (P, a) 
and (P', a') split R, with P, P' € M h , then (P, a)\R = (P', a')\R. □ 

An automaton is slow for Hopcroft if, for all executions of Hopcroft's algorithm, the 
splitters in the current waiting set either do not split or split in the same way: there is a 
unique class that is split into two classes, and always into the same two classes. 

More formally, at each step (W, V) of an execution, there is at most one class R in 
the current partition V that is split, and for all splitters (P, a) and (P', a') in W that split 
R, onehas (P,a)\R := (P',a')\R. 



The definition is close to the statement in Proposition 5.1 above, and indeed, one has 
the following property. 

Theorem 6.2. An automaton is slow for Moore if and only if it is slow for Hopcroft. 

Proof. Let A be a finite automaton. We first suppose that A is slow for Moore. We con- 
sider an execution of Hopcroft's algorithm, and we prove that each step of the execution 
that changes the partition produces a Moore partition. 

This holds for the initial configuration (W, V), since V — A4q. Assume that one has 
V = A4h for some configuration (W, V) and some h ^ 0. Let R be the class of M.h 
split by Moore's algorithm. 

Let S G W be the splitter chosen in Hopcroft's algorith m. T hen either S splits no 



class, and the partition remains equal to M. h or by Proposition |6.1| it splits the class R. In 
the second case, this class is split by 5 into two new classes, say R' and R" . The partition 
V = V \ {R.} U {R 1 , R"} is equal to M h+ v 

Conversely, suppose that A is slow for Hopcroft. We show that it is also slow for 
Moore by showing that the partition A4h+i has only one class more than Mh- For this, 



we use Proposition 4.2 which states that each class R in Aih is refined in A4h+i into the 
partition R, given by 

n = A A ( p > a )\ R - 

a£A PeM h 

We show by induction on the number of steps that, in any execution of Hopcroft's algo- 
rithm, V = A4h for some configuration (V, W). This holds for the initial configuration. 



Let (W, a) be some splitter in W. It follows from Proposition 5.1 that 



n > A A (W,a)\R. 

a£A (W,a)£W 
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Figure 5. An automaton over one letter recognizing the set of words of length at 
least n. 




b b b b 



Figure 6. An automaton recognizing the Dyck words of "height" at most n. 

Thus the partition 1Z is coarser than that the partition of R obtained by Hopcroft's algo- 
rithm. Since the automaton is slow for Hopcroft, the partition on the right-hand side has at 
most two elements. More precisely, there is exactly one class R that is split by Hopcroft's 
algorithm into two classes. Since Moore's partition Mh+i is coarser, it contains precisely 
these classes. This proves the claim. □ 

6.2 Examples 

Example 6.1. The simplest example is perhaps the automaton given in Figure || For 
^ h ^ n — 1, the partition Aih is composed of the class {0, . . . , n — h — 1}, and of the 
singleton classes {n — h}, {n — h + 1},. . . , {n}. At each step, the last state is split off 
from the class {0, . . . , n — h — 1}. 

Example 6.2. The automaton of Figure || recognizes the set of Dyck words w over 
{a, b} such that ^ \u\ a — \u\b ^ n for all prefixes u of w. The partition Aih, for 
^ h ^ n — 1, is composed of {0}, . . . , {h}, and {oo, h + 1, . . . , n}. At h = n, the state 
{oo} is separated from state n. 

Example 6.3. Let w — b\ ■ ■ ■ b n be a word of length n over the binary alphabet {0, 1}. 
We define an automaton A w over the unary alphabet {a} as follows. The state set of A w 
is {1, . . . , n} and the next state function is defined by i ■ a = i + 1 for i < n and n ■ a = 1. 
Note that the underlying labeled graph of A w is just a cycle of length n. The final states 
really depend on w. The set of final states of A w isF = {l^i^ri|6, = l}. We call 
such an automaton a cyclic automaton. 

For a binary word u, we define Q u to be the set of states of A w which are the starting 
positions of circular occurrences of u in w. If u is the empty word, then Q u is by con- 
vention the set Q of all states of A w . By definition, the set F of final states of A w is Qi 
while its complement F c is Q . 

Consider the automaton A w for w — 01001010 given in Fig. [7| The sets Qx, Q i an d 
Qu of states are respectively {2, 5, 7}, {1, 4, 6} and 0. If w is a Sturmian word, then the 
automaton Aw is slow. 
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Figure 7. Cyclic automaton A w for w = 01001010. Final states are circled. 



Slow automata are closely related to binary Sturmian trees. Consider indeed a finite 
automaton, for instance over a binary alphabet. To this automaton corresponds an infinite 
binary tree, composed of all paths in the automaton. The nodes of the tree are labeled with 
the states encountered on the path. For each integer h ^ 0, the number of distinct subtrees 
of height h is equal to the number of classes in the Moore partition M.t- It follows that 
the automaton is slow if and only if there are h + 1 distinct subtrees of height h for all h: 
this is precisely the definition of Sturmian trees, as given in [|l2|]. 

We consider now the problem of showing that the running time 0(n log n) for Hop- 
croft's algorithm on n-state automata is tight. The algorithm has a degree of freedom 
because, in each step of its main loop, it allows one to choose the splitter to be processed. 
Berstel and Carton [[d} introduced a family of finite a utom ata based on de Bruijn words. 
These are exactly the cyclic automata A w of Example 63 where w is a binary de Bruijn 
word. They showed that there exist some "unlucky" sequence of choices that slows down 
the computation to achieve the lower bound Vt(n log n). 

In the papers [ |20| ] and Castiglione, Restivo and Sciortino replace de Bruijn words 
by Fibonacci words. They observe that for these words, and more generally for all circu- 
lar standard Sturmian words, there is no more choice in Hopcroft's algorithm. Indeed, the 
waiting set always contains only one element. The uniqueness of the execution of Hop- 
croft's algorithm implies by definition that the associated cyclic automata for Sturmian 
words are slow. 

They show that, for Fibonacci words, the unique execution of Hopcroft's algorithm 
runs in time Q(jilogn), so that the worst-case behavior is achieved for the cyclic au- 
tomata of Fibonacci words. The computation is carried out explicitly, using connections 
between Fibonacci numbers and Lucas numbers. In [22|, they give a detailed analysis of 
the reduction process that is the basis of their computation, and they show that this process 
is isomorphic, for all standard Sturmian words, to the refinement process in Hopcroft's 
algorithm. 

In JTT||, the analysis of the running time of Hopcroft's algorithm is extended to cyclic 
automata of standard Sturmian words. It is shown that the directive sequences for which 
Hopcroft's algorithm has worst-case running time are those sequences (d%, d%, g?3, . . .) 
for which the sequence of geometric means ((p n ) n )n^u where p n = did?, ■ ■ ■ d n , is 
bounded. 
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7 Minimization by fusion 

In this section, we consider the minimization of automata by fusion of states. An impor- 
tant application of this method is the computation of the minimal automaton recognizing 
a given finite set of words. This is widely used in computational linguistics for the space- 
efficient representation of dictionaries. 

Let A be a deterministic automaton over the alphabet A, with set of states Q. The 
signature of a state p is the set of pairs (a,q) £ Ax Q such that p - a = q, together with a 
Boolean value denoting whether p is final or not. Two states p and q are called mergeable 
if and only if they have the same signature. The fusion or merge of two mergeable states 
p and q consists in replacing p and q by a single state. The state obtained by fusion of two 
mergeable states has the same signature. 

Minimization of an automaton by a sequence of fusion of states with the same signa- 
ture is not always possible. Consider the two-state automaton over the single letter a given 
in Figure ^Jwhich recognizes a*. It is not minimal. The signature of state 1 is +, (a, 2) 
and the signature of state 2 is +, (a, 1) (here "+" denotes an accepting state), so the states 
have different signatures and are not mergeable. 




a 



Figure 8. An automaton recognizing the set a* which can not be minimized by 
fusion of its states. 



7.1 Local automata 

M.-P. Beal and M. Crochemore [||] designed an algorithm for minimizing a special class 
of deterministic automata by a sequence of state mergings. These automata are called 
irreducible local automata. They occur quite naturally in symbolic dynamics. The running 
time of the algorithm is 0(min(m(n — r+ 1), to log n)), where to is the number of edges, 
n is the number of states, and r is the number of states of the minimized automaton. In 
particular, the algorithm is linear when the automaton is already minimal. Hopcroft's 
algorithm has running time 0(kn\ogn), where k is the size of the alphabet, and since 
kn ^ to, it is worse than Beal and Crochemore's algorithm. Moreover, their algorithm 
does not require the automaton to be complete. 

The automata considered here have several particular features. First, all states are 
both initial and final. Next, they are irreducible, that is, their underlying graph is strongly 
connected. Finally, the automata are local. By definition, this means that two distinct 
cycles carry different labels. This implies that the labels of a cycle are primitive words, 
since otherwise there exist different traversals of the cycle which have the same label. In 

the constructions and proofs are done for a more general family of automata called 
AFT (for automata of almost finite type). We sketch here the easier case of local automata. 
Since all states are final, two states p and q of an automaton are mergeable if and only 
if, for all letters a £ A, p ■ a is defined if and only if q ■ a and, if this is the case, then 
p ■ a = q ■ a. 



Minimization of automata 



19 



a b a a 




Figure 9. The automaton on the left is local, the automaton on the right is not 
because of the two loops labeled a, and because the label bb of the cycle through 1 
and 2 is not a primitive word. 



The basic proposition is the following. It shows that an irreducible local automaton 
can be minimized by a sequence of fusion of states. 

Proposition 7.1. If an irreducible local automaton is not minimal, then at least two of its 
states are mergeable. □ 

The minimization algorithm assumes that the alphabet is totally ordered. It uses the 
notion of partial signature. First, with each state q is associated the signature a{q) = 
aiPici2P2 ■ ■ ■ o-mPm, where {(ai,pi), . . . , (a m ,p m )} is the signature of q, and the se- 
quence is ordered by increasing value of the letters. Since all states are final, the Boolean 
indicator reporting this is omitted. A partial signature is any prefix a,\p\aiPi • • • Cb%P% of 
a signature. 

A first step consists in building a signature tree which represents the sets of states 
sharing a common partial signature. The root of the tree represents the set of all states, 
associated to the empty signature. A node representing the sets of states with a partial 
signature a\P\a<ip<i ■ ■ ■ cupi is the parent of the nodes representing the sets of states with 
a partial signature a\p\aipi ■ ■ ■ aiPiai + \Pi + \. As a consequence, leaves represent full 
signatures. All states that correspond to a leaf are mergeable. 

When mergeable states are detected in the signature tree, they can be merged. Then 
the signature tree has to be updated, and this is the difficult part of the algorithm. 

7.2 Bottom-up minimization 

In this section, all automata are finite, acyclic, deterministic and trim. A state p is called 
confluent if there are at least two edges in A ending in p. 

A trie is an automaton whose underlying graph is a tree. Thus an automaton is a trie 
if and only if it has no confluent state. 

Bottom-up minimization is the process of minimizing an acyclic automaton by a bot- 
tom-up traversal. In such a traversal, children of a node are treated before the node itself. 
During the traversal, equivalent states are detected and merged. The basic property of 
bottom-up minimization is that the check for (Nerode) equivalence reduces to equality of 
signatures. The critical point is to organize the states that are candidates in order to do 
this check efficiently. 

The bottom-up traversal itself may be organized in several ways, for instance as a 
depth-first search with the order of traversal of the children determined by the order on 
the labels of the edges. Another traversal is by increasing height, as done in Revuz's 
algorithm given next. 
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REVUZ(„4) 

for h = to Height(^) do 

S <- GetStatesForHeight(/i) > Compute states of height h 

SortSignatures(5) > Compute and sort signatures 

for s G S do > Merge mergeable states 
if s and s.next have the same signature then 

Merge (s, s.next) > Mergeable states are consecutive 

Figure 10. Revuz's minimization algorithm. 



One popular method for the construction of a minimal automaton for a given finite 
set of words consists in first building a trie for this set and then minimizing it. Daciuk et 
al. [ p7| ] propose an incremental version which avoids this intermediate construction. 

Recall that the signature of a state p is the set of pairs (a, q) such that p ■ a = q 
together with a Boolean value indicating whether the state is final or not. It is tacitly 
understood that the alphabet of the automaton is ordered. The signature of a state is 
usually considered as the ordered sequence of pairs, where the order is determined by 
the letters. It is important to observe that the signature of a state evolves when states are 
merged. As an example, the state 6 of the automaton on the left of Figure |ll] has signature 
+ , (a, 3), (b, 10), and the same state has signature +, (a, 3), (6, 7) in the automaton on the 
right of the figure. 

As already mentioned, if the minimization of the children of two states p and q has 
been done, then p and q are (Nerode) equivalent if and only if they have the same sig- 
nature. So the problem to be considered is the bookkeeping of signatures, that is the 
problem of detecting whether the signature of the currently considered state has already 
occured before. In practical implementations, this is done by hash coding the signatures. 
This allows one to perform the test in constant average time. One remarkable exception 
is Revuz's algorithm to be presented now, and its extension by Almeida and Zeitoun that 
we describe later. 

7.3 Revuz's algorithm 

Revuz p7| ] was the first to give an explicit description of a linear time implementation of 
the bottom-up minimization algorithm. The principle of the algorithm was also described 

by[0. 

Define the height of a state p in an acyclic automaton to be the length of the longest 
path starting at p. It is also the length of the longest word in the language of the subau- 
tomaton at p. Two equivalent states have the same height. Revuz's algorithm operates by 
increasing height. It is outlined in Figure |l0[ Heights may be computed in linear time by 
a bottom-up traversal. The lists of states of a given height are collected during this traver- 
sal. The signature of a state is easy to compute provided the edges starting in a state have 
been sorted (by a bucket sort for instance to remain within the linear time constraint). 
Sorting states by their signature again is done by a lexicographic sort. As for Moore's 
algorithm, the last step can by done by a simple scan of the list of states since states with 
equal signature are consecutive in the sorted list. 
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Figure 11. On the left a simple automaton: its nontrivial strongly connected com- 
ponents are the cycles 2, 4 and 3, 5, 9, 6. The minimization starts by merging 11, 12 
and 8, and also 10 and 7. This gives the automaton on the right. 

The whole algorithm can be implemented to run in time 0{m) for an automaton with 
m edges. 

Revuz's algorithm relies on a particular bottom-up traversal of the trie. This traversal 
is defined by increasing height of states, and it makes the check for equivalent states easier. 
With another method for checking signatures in mind, like hash coding, the algorithm may 
be organized in a different way. For instance, the traversal by heights can be replaced 
by a traversal by lexicographic order. The last item in the algorithm may be replaced by 
another check. Whenever a state has been found which must be in the minimal automaton, 
its hash code is registered. When the signature of a state is computed, one checks whether 
its hash code is registered. If not, it is added to the register, otherwise it is replaced by the 
hash code. 

Several implementations have been given in various packages. A comparison has been 
given in [Q. See also Table || below for numerical data. 

7.4 The algorithm of Almeida and Zeitoun 

Almeida and Zeitoun [|]| consider an extension of the bottom-up minimization algorithm 
to automata which contain only simple cycles. They describe a linear time algorithm for 
these automata. 

Let A be a finite trim automaton. We call it simple if every nontrivial strongly con- 
nected component is a simple cycle, that is if every vertex of the component has exactly 
one successor vertex in this component. The automaton given on the left of Figure |ll| 
is simple. Simple automata are interesting because they recognize exactly the bounded 
regular languages or, equivalently, the languages with polynomial growth. These are the 
simplest infinite regular languages. 

The starting point of the investigation of [Q] is the observation that minimization can 
be split into two parts: minimization of an acyclic automaton and minimization of the set 
of strongly connected components. There are three subproblems, namely (1) minimiza- 
tion of each strongly connected component, (2) identification and fusion of isomorphic 
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minimized strongly connected components, and (3) wrapping, which consists in merging 
states which are equivalent to a state in a strongly connected component, but which are 
not in this component. The authors show that if these subproblems can be solved in linear 
time, then, by a bottom-up algorithm which is a sophistication of Revuz's algorithm, the 
whole automaton can be minimized in linear time. Almeida and Zeitoun show how this 
can be done for simple automata. The outline of the algorithm is given in Figure [j~2|. 

The algorithm works as Revuz's algorithm as long as no nontrivial strongly connected 
components occur. In our example automaton, the states 8, 11 and 12 are merged, and the 
states 10 and 7 also are merged. This gives the automaton on the right of Figure |ll| . 

Then a cycle which has all its descendants minimized is checked for possible mini- 
mization. This is done as follows: the weak signature of a state p of a cycle is the signature 
obtained by replacing the name of its successor in the cycle by a dummy symbol, say □. 
In our example, the weak signatures of the states 3,5,9,6 are respectively: 

-a8bD . +aOb7 , -a8bD , +aOb7 . 

Here we write '+' when the state is final, and '— ' otherwise. 

It is easily seen that the cycle is minimal if and only if the word composed of the 
sequence of signatures is primitive. In our example, the word is not primitive since it 
is a square, and the cycle can be reduced by identifying states that are at corresponding 
positions in the word, that is states 5 and 6 can be merged, and states 3 and 9. This gives 
the automaton on the left of Figure [l3|. 

Similarly, in order to check whether two (primitive) cycles can be merged, one checks 
whether the words of their weak signatures are conjugate. In our example, the cycles 2, 4 
and 3, 5 have the signatures 

+aOb7 , -a8ba and - a8ba , + a ab7 . 

These words are conjugate and the corresponding states can be merged. This gives the 
automaton on the right of Figure [jj]. This automaton is minimal. 

A basic argument for preserving the linearity of the algorithm is the fact that the mini- 
mal conjugate of a word can be computed in linear time. This can be done for instance by 
Booth's algorithm (see [j^). Thus, testing whether a cycle is minimized takes time pro- 
portional to its length, and for each minimized cycle, a canonical representative, namely 
the unique minimal conjugate, which is a Lyndon word, can be computed in time pro- 
portional to its length. The equality of two cycles then reduces to the equality of the two 



AlmeidaZeitoun(.A) 

S <- ZeroHeight(„4) > States and cycles of height 
while S do 

MinimizeCycles (S) > Minimize each cycle 

MergeIsomorphicCycles(S') > Compute and sort signatures and merge 

Wrap(S) > Search states to wrap 

S <- NextHeight(.4, S) > Compute states for next height 



Figure 12. The algorithm of Almeida and Zeitoun. 
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Figure 13. The minimization continues by merging the states 5 and 6, and the states 
3 and 9. This gives the automaton on the left. The last step of minimization merges 
the states 2 and 5, and the states 3 and 4. 




Figure 14. The automaton on the left has one minimal cycle of height 1. By wrap- 
ping, states 6 and 8, and states 5 and 7 are merged, respectively, giving the automaton 
on the right. 



associated words. Finding isomorphic cycles is accomplished by a lexicographic ordering 
of the associated words, followed by a simple scan for equal words. 

A few words on wrapping: it may happen that states of distinct heights in a simple 
automaton are equivalent. An example is given in Figure [l4]. Indeed, states 6 and 8 have 
the same signature and therefore are mergeable but have height 1 and 0, respectively. This 
situation is typical: when states s and t are mergeable and have distinct heights, and t 
belongs to a minimized component of current height, then s is a singleton component on 
a path to the cycle of t. Wrapping consists in detecting these states, and in "winding" 
them around the cycle. In our example, both 6 and 5 are wrapped in the component of 7 
and 8. In the algorithm given above, a wrapping step is performed at each iteration, after 
the minimization of the states and the cycles and before computing the states and cycles 
of the next height. In our example, after the first iteration, states 3 and 4 are mergeable. 
A second wrapping step merges 3 and 4. These operations are reported in Figure [l5|. A 
careful implementation can realize all these operations in global linear time. 
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Figure 15. The automaton on the left has one minimal cycle of height 1. By wrap- 
ping, states 6 and 8, and states 5 and 7 are merged, respectively, giving the automaton 
on the right. 



7.5 Incremental minimization: the algorithm of Daciuk et al. 

The algorithm presented in [^7|| is an incremental algorithm for the construction of a 
minimal automaton for a given set of words that is lexicographically sorted. 

The algorithm is easy to implement and it is efficient: the construction of an automaton 
recognizing a typical dictionary is done in a few seconds. Table ^ was kindly communi- 
cated by Sebastien Paumier. It contains the space saving and the computation time for 
dictionaries of various languages. 

The algorithm described here is simple because the words are sorted. There exist 
other incremental algorithms for the case of unsorted sets. One of them will be described 
in the next section. Another algorithm, called semi-incremental because it requires a final 
minimization step, is given in J5l|]. 

We start with some notation. Let A — (Q,i,F)be a finite, acyclic, deterministic and 
trim automaton. We say that a word x is in the automaton A if i ■ x is defined. In other 
words, x is in A if x is a prefix of some word recognized by A. Let w be a word to be 
added to the set recognized by an automaton A. The factorization 

w = x ■ y , 

where x is the longest prefix of w which is in A, is called the prefix-suffix decomposition 
of w. The word x (resp. y) is the common prefix (resp. corresponding suffix) of w. 

One has x = e if either w — e or i ■ a is undefined, where a is the initial letter of w. 
Similarly, y = e if w itself is in A. If y ^ e and starts with the letter b, then i ■ xb = _L. 

The insertion of a word y at state p is an operation that is performed provided y = e or 
p ■ b = _L, where b is the initial letter of y. If y = e, the insertion simply consists in adding 
state p to the set F of final states. If y ^ e, set y — &1&2 • • • b m . The insertion consists 
in adding new states p\ , . . . , p m to Q, with the next state function defined by p ■ bi = p\ 
and pi^i ■ bi — pi for i = 2, . . . , m. Furthermore, p m is added to the set F of final states. 

Assume that the language recognized by A is not empty, and that the word w is lexi- 
cographically greater than all words in A. Then w is not in A. So the common prefix x 
of w is strictly shorter than w and the corresponding suffix y is nonempty. 

The incremental algorithm works at follows. At each step, a new word w that is 
lexicographically greater than all previous ones is inserted in the current automaton A. 
First, the prefix-suffix decomposition w = xy of w, and the state q — i-x are computed. 
Then the segment starting at q of the path carrying the suffix y' of the previously inserted 
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File 


Lines 


Text file 


Automaton 


Time 


States 


Trans. 


Size 


Revuz 


Daciuk 


delaf-de 


189878 


12.5Mb 


57165 


103362 


1.45Mb 


4.22s 


4.44s 


delaf-en 


296637 


13.2Mb 


109965 


224268 


2.86Mb 


5.94s 


6.77s 


delaf-es 


638785 


35.4Mb 


56717 


117417 


1.82Mb 


10.61s 


11.28s 


delaf-fi 


256787 


24.6Mb 


124843 


133288 


4.14Mb 


6.40s 


7.02s 


delaf-fr 


687645 


38.7Mb 


109466 


240409 


3.32Mb 


13.03s 


14.14s 


delaf-gr 


1288218 


83.8Mb 


228405 


442977 


7.83Mb 


28.33s 


31.02s 


delaf-it 


611987 


35.9Mb 


64581 


161718 


1.95Mb 


10.43s 


11.46s 


delaf-no 


366367 


23.3Mb 


75104 


166387 


2.15Mb 


6.86s 


7.44s 


delaf-pl 


59468 


3.8Mb 


14128 


20726 


502Kb 


1.19s 


1.30s 


delaf-pt 


454241 


24.8Mb 


47440 


115694 


1.4Mb 


7.87s 


8.45s 


delaf-ru 


152565 


10.8Mb 


23867 


35966 


926Kb 


2.95s 


3.17s 


delaf-th 


33551 


851Kb 


36123 


61357 


925Kb 


0.93s 


1.14s 



Table 2. Running time and space requirement for the computation of minimal au- 
tomata (communication of Sebastien Paumier). 



DaciukEtAl(^) 
for all w do 

[x, y) <- PrefSuffDecomp(w) 
q 4— i ■ x 

MinimizeLastPath(<7) 
AddPath(<7, y) 



> Words are given in lexicographic order 

> x is the longest prefix of w in A 

> q is the state reached by reading x 

> Minimize the states on this path 

> Adds a path starting in q and carrying y 



Figure 16. The incremental algorithm of Daciuk et al. 



word w' is minimized by merging states with the same signature. Finally, the suffix y is 
inserted at state q. The algorithm is given in Figure [l6|. 

The second step deserves a more detailed description. We observe first that the word 
x of the prefix-suffix decomposition w = xy of w is in fact the greatest common prefix 
of w' and w. Indeed, the word a; is a prefix of some word recognized by A (here A is the 
automaton before adding w), and since w' is the greatest word in A, the word x is a prefix 
of w' . Thus x is a common prefix of w 1 and w. Next, if x' is a common prefix of w' and 
w, then x' is in A because it is a prefix of w', and consequently x' is a prefix of x because 
x is the longest prefix of w in A. This shows the claim. 

There are two cases for the merge. If w' is a prefix of w, then w' — x. In this case, 
there is no minimization to be performed. 

If w' is not a prefix of w, then the paths for w' and for w share a common initial 
segment carrying the prefix x, from the initial state to state q = i ■ x. The minimization 

concerns the states on the path q t' carrying the suffix y' of the factorization w = xy' 
of w'. Each of the states in this path, except the state q, will never be visited again in any 
insertion that may follow, so they can be merged with previous states. 
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(a) (b) 

Figure 17. (a). The automaton for aa, aba. (b). The automaton for aa, aba, ba. Here 
state 4 has been merged with state 2. 





(a) (b) 

Figure 18. (a). The automaton for aa, aba, ba, aba, bba. (b). The automaton for 
aa, aba, ba, aba, bba, be. After inserting be, states 8 and 2 are merged, and then states 
7 and 2. 



Example 7.1. We consider the sequence of words (aa, aba, ba, bba, be). The first two 
words give the automaton of Figure [lj(a). Adding the word ba permits the merge of 
states 2 and 4. The resulting automaton is given in Figure [n|(b). After inserting bba, there 
is a merge of states 6 and 2, see Figure p~8](a). 



8 Dynamic minimization 

Dynamic minimization is the process of maintaining an automaton minimal when inser- 
tions or deletions are performed. 

A solution for adding and for removing a word was proposed by Carrasco and Forcada 
jl^ ] . It consists in an adaptation of the usual textbook constructions for intersection and 
complement to the special case where one of the languages is a single word. It appears 
that the finiteness of the language L plays no special role, so we assume here that it is 
regular, not necessarily finite. The construction for adding a word has also been proposed 
in [ftjfl, and in [[27j| for acyclic automata. An extension to general automata, and several 
other issues, are discussed in [p6|. 

We consider here, for lack of space, only deletion of a word from the set recognized 
by an automaton, and minimization of the new automaton. 

Let A — (Q, i, T) be the minimal automaton recognizing a language L over the al- 
phabet A, and let wbea word in L. Denote by A w the minimal automaton recognizing 
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a 




a, b 



b 

Figure 19. The minimal automaton recognizing the complement of the word abab. 
Only the state abab is not final. 

the complement A* \ w. The automaton has n + 2 states, with n = \w\. Among them, 
there are n + 1 states that are identified with the set P of prefixes of w. The last state is a 
sink state denoted _!_. An example is given in Figure [l9|. 

The language L \ w is equal to L D (A* \ w), so it is recognized by the trimmed part 
B of the product automaton A x A tu . Its initial state is (i, e), and its states are of three 



• intact states: these are states of the form (q, _L) with q E Q. They are called 
so because the language recognized at (q, _L) in B is the same as the language 
recognized at q in A: Lg(q, _L) = L^(q). 

• cloned states: these are accessible states (q, x) with x e P, so x is a prefix of w. 
Since we require these states to be accessible, one has q = i ■ x in A, and there is 
one such state for each prefix. The next-state function on these states is defined by 



Observe that (i ■ w, _L) is an intact state because w is assumed to be recognized 
by A. 

• useless states: these are all states that are removed when trimming the product 
automaton. 

Trimming consists here in removing the state (i, _L) if it is no longer accessible, and the 
states reachable only from this state. For this, one follows the path defined by w and 
starting in (i, _L) and removes the states until one reaches a confluent state (that has at 
least two incoming edges). The automaton obtained is minimal. 

The whole construction finally consists in keeping the initial automaton, by renaming 
a state q as (q, _L), adding a cloned path, and removing state (i, _L) if it is no longer 
accessible, and the states reachable only from this state. 

Of course, one may also use the textbook construction directly, that is without taking 
advantage of the existence of the automaton given at the beginning. For this, one starts at 
the new initial state (i, e) and one builds only the accessible part of the product automaton. 
The method has complexity 0(n + \w\), where n is the number of states of the initial 
automaton, whereas the previous method has only complexity 0(\w\). 

Example 8.1. The automaton given in Figure |T] recognizes the language L — (ab) + U 
{abc, acb}. The direct product with the automaton of Figure |l9| is shown in Figure ^2| 
Observe that there are intact states that are not accessible from the new initial state (0, e). 



kinds. 




(q ■ a, xa) if xa G P, 
(q ■ a, _L) otherwise. 
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RemoveIncremental(w, A) 
A' <- AddClonedPath(iu, A) 
Trim (.A') 



> Add a fresh path for w in A 

> Return trimmed automaton 



AddClonedPath(<2i • • • a n , A) 
Po <- Inijial(A); qo Clone(p ) > Add a fresh initial state q 
for i = 1 to n do 

Pi *-Pi-i ■ en; Qi <- Clone fe) 

qi-i ■ di <r- % 
SetFinal((7„, false) 

Figure 20. Removing the word w from the language recognized by A. 



> q,; inherits the transitions of pi 

> This edge is redirected 




Figure 21. The minimal automaton recognizing the language L = (ab) + U 
{abc, acb}. 



The minimal automaton is shown in Figure E3. 



9 Extensions and special cases 

In this section, we consider extensions of the minimization problem to other classes of 
automata. The most important problem is to find a minimal nondeterministic automaton 




Figure 22. The automaton recognizing the language L = (ab) + U {abc, acb} \ 
{abab}. There are still unreachable states (shown in gray). 
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(I 




Figure 23. The minimal automaton for the language L = (ab) + U {abc,acb} \ 



recognizing a given regular language. Other problems, not considered here, concern sets 
of infinite words and the minimization of their accepting devices, and the use of other 
kinds of automata known to be equivalent with respect to their accepting capability, such 
as two-way automata or alternating automata, see for instance [ |46| ] . 



We briefly mention here special cases where minimization plays a role. It is well known 
that string matching is closely related to the construction of particular automata. If w 
is a nonempty word over an alphabet A, then searching for all occurrences of w as a 
factor in a text t is equivalent to computing all prefixes of t ending in w, and hence 
to determining all prefixes of t which are in the regular language A*w. The minimal 
automaton recognizing A*w has n + 1 states, where n = \w\, and can be constructed 
in linear time. The automaton has many interesting properties. For instance there are at 
most 2n edges, when one does not count edges ending in the initial state. This is due to 
Imre Simon, see also [p3|]. For a general exposition, see e.g. [fijj], Extension of string 
matching to a finite set X of patterns has been done by Aho and Corasick. The associated 
automaton is called the pattern matching machine; it can be computed in time linear in 
the sum of the lengths of the words in X. See again p4|. However, this automaton is 
not minimal in general. Indeed, the number of states is the number of distinct prefixes 
of words in X, and this may be greater than than the number of states in the minimal 
automaton (consider for example the set X = {ab, bb} over the alphabet A = {a, b}). 
There are some investigations on the complexity of minimizing Aho-Corasick automata, 



Another famous minimal automaton is the suffix automaton. This is the minimal au- 
tomaton recognizing all suffixes of a given word. The number of states of the suffix 
automaton of a word of length n is less than 2n, and the number of its edges is less than 
3n. Algorithms for constructing suffix automata in linear time have been given in [Hj 



{abab}. 



9.1 Special automata 



see 




9.2 Nondeterministic automata 



A nondeterministic automaton is minimal if it has the minimal number of states among all 
automata recognizing the same language. Nondeterministic automata are not unique. In 
Figure we give two non-isomorphic nondeterministic automata which are both smaller 




than the minimal deterministic automaton recognizing the same language. This language 
is a(b* U c*)ab + . The example is derived from an automaton given in [|J|. 

One might ask if there are simple conditions on the automata or on the language that 
ensure that the minimal nondeterministic automaton is unique. For instance, the automata 
of Figure ^5] both recognize the same language, but the second has a particular property 
that we will describe now. The uniqueness of the minimal automaton in the deterministic 



a,b a,b 




a, b 

Figure 25. Two nondeterministic automata recognizing the set of words ending with 
the letter a. 



case is related to the fact that the futures of the states of such an automaton are pairwise 
distinct, and that each future is some left quotient of the language: for each state q, the 
language L q (A) is equal to a set y~ 1 L, for some word y. 

This characterization has been the starting point for investigating similar properties of 
nondeterministic automata. Let us call a (nondeterministic) automaton a residual automa- 
ton if the future of its states are left quotients of the language; it has been shown in [ f30| | 
that, among all residual automata recognizing a given language, there is a unique residual 
automaton having a minimal number of states; moreover, this automaton is characterized 
by the fact that the set of its futures is the set of the prime left quotients of the language, 
a left quotient being prime if it is not the union of other nonempty left quotients. For 
instance, the automaton on the right of Figure [B] has this property, since Lq = {a, b}*a 
and L\ = a _1 Lo = e U {a, b}*a and there are no other nonempty left quotients. The 
automaton on the left of Figure [B] is not residual since the future of state 1 is not a left 
quotient. 

The problem of converting a given nondeterministic automaton into a minimal non- 
deterministic automaton is NP-hard, even over a unary alphabet [pq], This applies also 
to unambiguous automata [p3[], In jl5[], these results have been extended as follows. The 
authors define a class (5NFA of automata that are unambiguous, have at most two compu- 
tations for each string, and have at most one state with two outgoing transitions carrying 
the same letter. They show that minimization is NP-hard for all classes of finite automata 
that include <5NFA, and they show that these hardness results can also be adapted to the 
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setting of unambiguous automata that can non-deterministically choose between two start 
states, but are deterministic everywhere else. 

Even approximating minimization of nondeterministic automata is intractable, see 
[pi]]. There is an algebraic framework that allows one to represent and to compute all 
automata recognizing agiven regular language. The state of this theory, that goes back to 
Kameda and Weiner p9jj, has been described by Lombardy and Sakarovitch in a recent 
survey paper . 

There is a well-known exponential blow-up from nondeterministic automata to deter- 
ministic ones. The usual textbook example, already given in the first section (the automa- 
ton on the left in Figure [l]) shows that this blow-up holds also for unambiguous automata, 
even if there is only one edge that causes the nondeterminism. 

It has been shown that any value of blow-up can be obtained, in the following sense 
[p7[]: for all integers n, N with n ^ N ^ 2™, there exists a minimal nondeterministic au- 
tomaton with n states over a four-letter alphabet whose equivalent minimal deterministic 
automaton has exactly N states. This was improved to ternary alphabets [38 1. 
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Abstract. This chapter is concerned with the design and analysis of algorithms for minimizing 
finite automata. Getting a minimal automaton is a fundamental issue in the use and implementation 
of finite automata tools in frameworks like text processing, image analysis, linguistic computer 
science, and many other applications. 

Although minimization algorithms are rather old, there were recently some new developments 
which are explained or sketched in this chapter. 

There are two main families of minimization algorithms. The first are by a sequence of refine- 
ments of a partition of the set of states, the second by a sequence of fusions or merges of states. 
Hopcroft's and Moore's algorithms belong to the first family. The linear-time minimization of 
acyclic automata of Revuz belongs to the second family. 

We describe, prove and analyze Moore's and Hopcroft's algorithms. One of our studies is 
the comparison of these algorithms. It appears that they quite different both in behavior and in 
complexity. In particular, we show that it is not possible to simulate the computations of one of 
the algorithm by the other. We also state the results known about average complexity of both 
algorithms. 

We describe the minimization algorithm by fusion for acyclic automata, and for a more gen- 
eral class of automata, namely those recognizing regular languages having polynomially bounded 
number of words. 

Finally, we consider briefly incremental algorithms for building minimal automata for finite sets 
of words. We consider the updating of a minimal automaton when a word is added or removed. 
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